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Abstract 

This research investigates the implementation of the programming language Alice to teach computer 
programming logic to computer information systems students. Alice has been implemented in other 
university settings and has been reported to have many benefits including object-oriented concepts 
and an engaging and fun learning environment. In this study, students were surveyed on their opi¬ 
nions about the effectiveness of the Alice environment and were compared to responses of students 
who had been taught in a more traditional environment using flowcharts and pseudocode. Analysis of 
data revealed there were no statistically significant differences in responses, with both groups report¬ 
ing high levels of satisfaction with their respective learning experience. Discussion includes ways to 
potentially improve the implementation of Alice and its benefits in the classroom. 
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1. INTRODUCTION 

Researchers have been discussing the best ap¬ 
proach to teaching the introductory computer 
class for almost as long as computing languages 
have been around (Pears, et a I., 2007). The 
first programming class is a fundamental part of 
the learning process for new programmers but 
many students find the class to be difficult and 
uninteresting, leading to high rates of failure or 
drop rates. As a result, faculty members in 
computer science and information systems pro¬ 
grams continually look for better ways to instill 
basic programming skills into tomorrow's pro¬ 
grammers. 

From studies of motivation and cognition to ex¬ 
perimentation with new programming environ¬ 
ments, researchers approach the problem from 
many directions. One way that has garnered 
some attention over the past few years has been 
changing the environment in which the basic 
constructs of programming are taught. Some 


programs have implemented introductory 
courses taught in scripting languages such as 
Python (Nikula, et al., 2007) while others have 
adopted new environments such as the pro¬ 
gramming language Alice. 

This study looks at the use of Alice in the class¬ 
room and compares student attitudes about 
programming confidence, success, applicability, 
and preparedness. These attitudes are com¬ 
pared with students in the same program who 
were taught programming fundamentals using 
more traditional methods including flowcharts 
and pseudocode only. 

2. BACKGROUND 

The concern about introductory programming 
courses begins with the widely reported failure 
rates of first time programmers that range from 
30 to 60% in computer science programs (Deh- 
nadi & Bornat, 2006). Researchers speculate 
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that there are many reasons for failure and suc¬ 
cess in the programs that range from cognitive 
to motivational. In his thesis The Motivation of 
Students of Programming, Jenkins (2001) post¬ 
ulates that the reasons for failure or success 
stem from who the student is, what the teacher 
does, and what the student does. In other 
words, there are many factors that come into 
play when deciding why students fail or succeed 
in these classes. When asking students why 
they believe they failed an introductory comput¬ 
er class, Hawi (2010) found that most students 
cited learning strategies and teaching methods 
as the main reasons for poor performance, fol¬ 
lowed closely by lack of appropriate effort on the 
part of the student. 

When examining learning strategies, we must 
look at the way many introductory courses are 
taught. Many times the examples and strategies 
used to explain basic concepts are the same 
teaching methods that have been around for 
years. Although the iconic "hello world" example 
lives on, it may not be the most exciting or rele¬ 
vant programming for students today. Combine 
these strategies with a learning environment 
that requires high levels of abstraction in a com¬ 
plex modern language and many students will 
not or cannot comprehend the material being 
presented (Nikula, et al., 2007). 

Given these many barriers, there are still many 
students who succeed in the introductory pro¬ 
gramming course and are highly motivated to 
perform well. Law, Lee, and Yu ( 2010) discov¬ 
ered a complex relationship between individual 
attitude, direction, social pressure and competi¬ 
tion, and reward or recognition. In other studies, 
there was a significant importance placed on the 
importance of value to the student and the ex¬ 
pectancy of the outcomes, although this motiva¬ 
tion was found to be short lived and affected 
mostly the student's motivation in that one 
course and did not extend to the rest of the aca¬ 
demic or professional career (Jenkins, 2001). 
Keeping students interested in programming is 
difficult because many come to the class with 
preconceived ideas of difficulty and many view 
programming as boring. The job of changing 
these attitudes and motivating students to suc¬ 
ceed is daunting to many faculty members but 
takes on greater importance as enrollment in 
computer science and information systems pro¬ 
grams continues to decline while the demand for 
skilled workers increases (Kelleher & Pausch, 
2007 ). 
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Alice Programming Environment 

In order to motivate students and make first 
programming experiences more interesting for 
students, some universities have integrated the 
software program Alice. Alice was developed by 
Randy Pausch at Carnegie Mellon in an attempt 
to encourage younger girls to take an interest in 
computer programming. Kelleher and Pausch 
(2007) reported success in motivating girls 
through the storytelling element with an easy 
user interface and state that in addition to sim¬ 
plifying the learning environment, tools need to 
be developed that attract students to the pro¬ 
gramming field. 

In the university environment, the implementa¬ 
tion of Alice has taken many forms from mod¬ 
ules in beginning programming classes to entire 
semester courses teaching fundamental con¬ 
cepts through Alice. In addition to tangible 
strengths of presenting object oriented concepts, 
intangible benefits of Alice include better visual 
rewards and reinforcement, intrinsic motivation 
from storytelling aspect, self-confidence, and 
increased interest in computers (Goulet & Slater, 
2009). Many researchers have reported success 
stories in the use of Alice and resulted in in¬ 
creases in positive attitudes towards program¬ 
ming in an introductory computer class for non¬ 
majors. Many comments from students in the 
class commented on the creativity and the fun 
nature of the programming environment. 
(Courte, Howard & Bishop-Clark, 2006). In a 
study where Alice was used in pair programming 
exercises, students found Alice to be "simple" 
and the success of the students increased when 
students worked in pairs with the tool. (Howard, 
Evans, Courte, & Bishop-Clark, 2009). 

There are however drawbacks to the use of the 
tool in the university environment. Some critics 
believe it is not appropriate for undergraduate 
learners because the program is not a fully de¬ 
veloped program that is considered "useful" in 
industry (McKenzie, 2009). Because the focus of 
Alice is storytelling using 3D visual objects, oth¬ 
er critics do not believe it is an appropriate tool 
to use particularly when teaching business pro¬ 
gramming because examples and programming 
problems do not lend themselves to the envi¬ 
ronment. 
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3. PROBLEM 

Because learning many times occurs on the 
edges of what we already know and prior know¬ 
ledge can affect the success of a first year pro¬ 
grammer, Computer Information Systems (CIS) 
majors at Tarleton State university begin their 
programming sequence with an introductory 
class focusing on programming logic (Robins, 
2010). At times the course was offered in the 
same semester as the first programming lan¬ 
guages and other times it preceded enrollment 
in the first language. As interest in object- 
oriented languages increased and enrollments in 
procedural languages such as COBOL began to 
decline, the logic course began to deviate 
somewhat from the concepts being presented in 
the first language course. At the same time, 
enrollment overall began to decline in the pro¬ 
gram and fewer student were exploring CIS as a 
degree option. Anecdotally, students in the 
course were bored and attendance was dismal. 
As state budgets constricted and the legislature 
mandated lower core curricula, computer literacy 
courses were dropped from many degree plans 
and the CIS department had even fewer oppor¬ 
tunities to attract new students. 

Because of these factors, faculty began to ex¬ 
plore new ways to teach the logic course to bet¬ 
ter prepare students for their first programming 
class, to engage students in the process, and 
hopefully attract more students to the pro¬ 
gramming degree. At the time, many other uni¬ 
versities were reporting moderate success with 
Alice and textbooks were developed to facilitate 
the formal teaching of Alice. Faculty hoped that 
Alice would be the tool to teach object-oriented 
fundamentals in a fun, engaging manner that 
would motivate students to learn and possible 
attract more majors. 

Alice has been used to teach programming logic 
since 2008 and approximately half of the stu¬ 
dents in the program were instructed using the 
old method and newer students were instructed 
using the Alice environment. All students, 
whether they concentrate their studies in pro¬ 
gramming or network administration, are re¬ 
quired to complete the course before beginning 
their first programming language. Currently, the 
students can choose to take JAVA or Visual Basic 
as their first programming course. To determine 
the student's perceptions of the changes and the 
effectiveness in meeting some of the depart¬ 
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mental goals, this research project was underta¬ 
ken. 

4. METHODS 

A survey (Appendix A) was administered to stu¬ 
dents enrolled in CIS classes beyond the logic 
course. The survey was conducted face to face 
and was not administered to students in online 
courses. Students were asked to provide some 
demographic information as well as indicate the 
semester in which they took the logic course. 
Then, students were asked to provide their opi¬ 
nions on their experience in the logic course. In 
addition to an open ended question allowing 
students to provide additional comments, ques¬ 
tions on the survey covered the following con¬ 
cepts: 

• Relatedness to concepts in first pro¬ 
gramming class 

• Confidence in abilities to program 

• Enjoyment or fun in programming 

• Preparedness for first programming class 

• Improvement in problem solving 

• Influence on pursuing a CIS degree 

• Applicability in other CIS classes 

• Improvement of programming skills 

• Perceived value of class 

• Recommendation of class to others 

Students were asked to indicate their agreement 
with positively phrased questions covering these 
concepts on a Likert scale from 1 to 5. 

Forty-three students completed the survey in 
the spring of 2010. Of the students who partici¬ 
pated in the survey, three were female and for¬ 
ty-one were male. Thirty-eight of the students 
were aged 18-25, with four students in the age 
group 25-34 and one student over 35 years of 
age. Twenty-seven students indicated a concen¬ 
tration in network administration, ten in systems 
development, and six did not indicate their area 
of concentration. Of the students responding, 
20 completed the logic course before the imple¬ 
mentation of Alice and 23 used Alice. 

Survey results were compiled in Microsoft Excel 
and imported into SPSS for analysis. The res¬ 
ponses to questions about student attitude were 
compared between students who were taught 
using the older method and students who were 
taught using the Alice environment. Because of 
the small population size and the use of ordinal 
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data, tests were conducted to determine differ¬ 
ences in population using the Mann-Whitney 
test. 

5. RESULTS 

When analyzing the data, each question was 
examined to determine if differences existed 
between the two groups of students who had 
been instructed in different learning environ¬ 
ments (see Appendix B). Surprisingly, on all ten 
questions there are no statistically significant 
differences in the responses given by students in 
the two different groups. 

In addition to the differences in learning envi¬ 
ronment, other factors such as first program¬ 
ming language and concentration were also ana¬ 
lyzed to see if these had an impact on res¬ 
ponses. An analysis of responses between stu¬ 
dents who had used Alice in the introductory 
program and concentrated their studies in net¬ 
work administration and students who were in¬ 
terested in programming revealed no significant 
differences in their attitudes towards Alice as a 
learning tool, (see Appendix C) 

In order to determine if responses may have 
been affected by choice of first programming 
language when determining efficacy of Alice as a 
teaching tool, data were analyzed and compared 
students who took Microsoft Visual Basic against 
those students who enrolled in Java for their first 
programming language (see Appendix D). How¬ 
ever, there were no significant findings between 
responses except for one. Students who took 
Visual Basic as their first programming languag¬ 
es provided more positive responses when asked 
if the logic class had influenced their decision to 
pursue a CIS degree (U = 7.5, p=>02). 

6. DISCUSSION 

When examining the data, the results indicate 
that the inclusion of Alice has not brought much 
change in students' perceptions of the pro¬ 
gramming and logic course. However, there are 
many other variables to consider when examin¬ 
ing this issue. Previous iterations of the logic 
course had been taught for many years previous 
to the respondents' enrollment in the course. 
The curriculum and the preparation had been 
fine tuned by many of the professors teaching 
the course and was many times tied directly to 
the problems being presented in the program¬ 
ming courses. 
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As the tool continues to be used in the logic 
class, perhaps better coordination between in¬ 
structors teaching the logic tool and instructors 
teaching programming languages could leverage 
some of the advantages Alice has been previous¬ 
ly been reported to provide. Also, as with any 
new software program, installation issues and 
program functionality issues may have had an 
impact on attitudes towards Alice. Some stu¬ 
dents reported in the comment section of the 
survey that they had many technical issues with 
installation and stability of Alice that frustrated 
them throughout the process. Once these tech¬ 
nical issues are resolved or perhaps better in¬ 
stallation instructions are provided for students 
there may be an in improvements to students' 
attitudes toward the tool. 

Overall, students appear satisfied with the use¬ 
fulness and applicability of concepts learned in 
the logic and programming class regardless of 
the tool implemented to teach the course. A 
comparison between the two groups does not 
necessarily discount the use of Alice as a tool 
but demonstrates that perhaps the potential of 
the tool has not yet been maximized. Anecdo¬ 
tally, students have reported a disconnect be¬ 
tween the types of problems being solved in 
Alice and the business problems being presented 
in the first programming course. Perhaps a 
blending of traditional flowcharting and pseudo¬ 
code for business problems combined with the 
fun and engaging environment of Alice to teach 
basic concepts and logic structures may increase 
relevancy and value for students in the course. 
For example, an initial assignment in Alice may 
introduce students to the concept of iterations 
by having a penguin flap its wings in Alice. 
Then, once the logical concept has been pre¬ 
sented, a second example relating to business 
may help students bridge the gap between the 
3D storytelling environment into a framework 
that will more closely mimic the types of prob¬ 
lems they will encounter in their first program¬ 
ming class and ultimately in their career. 

Further research needs to be conducted in this 
area to go beyond the students' expectations 
and attitudes towards the tool. Measures of 
success whether they be tied to grades in pro¬ 
gramming classes or performance in capstone 
courses needs to be analyzed as well. In depth 
analysis of student learning on various concepts 
may reveal strengths and weaknesses of both 
environments that may lead to blended uses of 
tools to teach different concepts. For example, 
research may reveal that Alice is very effective 
in presenting the ideas of objects but might not 
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be as strong at presenting the logical structures 
of programming such as decision structures or 
looping structures. 

In addition to its efficacy in teaching introducto¬ 
ry concepts, further research needs to be con¬ 
ducted on the usefulness of the tool in attracting 
and retaining students. Because Alice was in¬ 
itially designed to generate interest in younger 
girls through storytelling and interactivity, stu¬ 
dies need to be done to determine if this is in¬ 
deed drawing more women to a field that is tra¬ 
ditionally male and also if the concepts of story¬ 
telling and interactivity appeal to male students 
as much as they might to female students. Un¬ 
fortunately, only three respondents in this study 
were female and did not allow for analysis of this 
type. 

In conclusion, the implementation of Alice in the 
programming logic class was not a failure. Stu¬ 
dents reported high levels of satisfaction with 
the tool similar to those reported by students in 
previous, more traditional introductory course. 
With improvements to the content presented 
and with tweaks to the use of the tool, both 
technically and pedagogically, there is hope that 
many of the benefits of Alice may be realized in 
future course offerings. 
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Appendices 


Appendix A - Survey Questions 

Students were asked to provide their responses on a 5 point Likert scale from 1 (Strongly Disagree) to 
5 (Strongly Agree) 

1. The concepts I learned in the programming and logic class (CIS110) directly related to concepts 
used in my first programming class,. 

2. The programming and logic class increased my confidence in my abilities to program. 

3. The programming and logic class made programming seem like fun. 

4. The programming and logic class prepared me for my first programming class. 

5. The programming and logic class helped me solve problems in my first programming class. 

6. The programming and logic class influenced me to pursue a CIS degree. 

7. The programming and logic class has helped me in other CIS classes. 

8. The programming and logic class made me a better a programmer. 

9. The programming and logic class was a valuable experience. 

10. I would recommend the programming and logic class to new CIS students. 


Appendix B - Analysis Results of Responses Between Students in Different Learning Envi¬ 
ronments 


Question 

Environment 

N 

Mean Rank 

Sum of 
Ranks 

Mann- 
Whitney U 

Significance 

1 

Alice 

20 

22.00 

440.00 

230.000 

1.000 


Non-Alice 

23 

22.00 

506.00 



2 

Alice 

20 

21.30 

426.00 

216.000 

.724 


Non-Alice 

23 

22.61 

520.00 



3 

Alice 

20 

20.85 

417.00 

207.000 

.564 


Non-Alice 

23 

23.00 

529.00 



4 

Alice 

20 

23.34 

443.50 

183.500 

.361 


Non-Alice 

23 

19.98 

459.50 



5 

Alice 

20 

22.12 

442.50 

227.500 

.950 


Non-Alice 

23 

21.89 

503.50 



6 

Alice 

20 

23.00 

460.00 

210.00 

.612 


Non-Alice 

23 

21.13 

486.00 



7 

Alice 

20 

21.25 

425.00 

215.00 

.708 


Non-Alice 

23 

22.65 

521.00 



8 

Alice 

20 

21.22 

424.50 

214.500 

.698 


Non-Alice 

23 

22.67 

521.50 



9 

Alice 

20 

21.48 

429.50 

219.500 

.791 


Non-Alice 

23 

22.46 

516.50 



10 

Alice 

20 

20.70 

414.00 

204.00 

.507 


Non-Alice 

23 

23.13 

532.00 
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Appendix C - Analysis of Responses of Students in Alice Environment by First Programming 
Language 


Question 

1 st Pro¬ 
gramming 
Languaqe 

N 

Mean Rank 

Sum of 
Ranks 

Mann- 
Whitney U 

Significance 

1 

Java 

12 

9.50 

114.00 

36.000 

.650 


Visual Basic 

7 

10.86 

76.00 



2 

Java 

12 

9.04 

108.50 

30.500 

.340 


Visual Basic 

7 

11.64 

81.50 



3 

Java 

12 

8.33 

100.00 

22.000 

.100 


Visual Basic 

7 

12.86 

90.00 



4 

Java 

12 

8.58 

103.00 

25.000 

.167 


Visual Basic 

7 

12.43 

87.00 



5 

Java 

12 

9.21 

110.50 

32.500 

.432 


Visual Basic 

7 

11.36 

79.50 



6 

Java 

12 

7.12 

85.50 

7.500 

.002 


Visual Basic 

7 

14.93 

104.50 



7 

Java 

12 

8.75 

105.00 

27.000 

.227 


Visual Basic 

7 

12.14 

85.00 



8 

Java 

12 

9.83 

118.00 

40.000 

.902 


Visual Basic 

7 

10.29 

72.00 



9 

Java 

12 

9.25 

111.00 

33.000 

.482 


Visual Basic 

7 

11.29 

79.00 



10 

Java 

12 

9.08 

109.00 

31.000 

.384 


Visual Basic 

7 

11.57 

81.00 




Appendix D - Analysis of Responses of Students in Alice Environment by Concentration 


Question 

Concentration 

N 

Mean Rank 

Sum of 
Ranks 

Mann- 
Whitney U 

Significance 

1 

Networking 

11 

9.00 

99.00 

33.000 

.659 


Systems Dev 

7 

10.29 

72.00 



2 

Networking 

11 

9.14 

100.50 

34.500 

.724 


Systems Dev 

7 

10.07 

70.50 



3 

Networking 

11 

8.41 

92.50 

26.500 

.285 


Systems Dev 

7 

11.21 

78.50 



4 

Networking 

11 

9.00 

99.00 

33.000 

.659 


Systems Dev 

7 

10.29 

72.00 



5 

Networking 

11 

8.64 

95.00 

29.000 

.425 


Systems Dev 

7 

10.86 

76.00 



6 

Networking 

11 

9.09 

100.00 

34.000 

.724 


Systems Dev 

7 

10.14 

71.00 



7 

Networking 

11 

8.91 

98.00 

32.000 

.596 


Systems Dev 

7 

10.43 

73.00 



8 

Networking 

11 

9.14 

100.50 

34.500 

.724 


Systems Dev 

7 

10.07 

70.50 



9 

Networking 

11 

8.68 

95.50 

29.500 

.425 


Systems Dev 

7 

10.79 

75.50 



10 

Networking 

11 

8.23 

90.50 

24.500 

.211 


Systems Dev 

7 

11.50 

80.50 
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